A programmable execution service (“PES”) can provide computing resources for executing applications on a permanent or an as-needed basis. The computing resources provided by a PES may include various types of resources, such as data processing resources, data storage resources, data communication resources, and the like. Each type of computing resource may be general-purpose or may be available in a number of specific configurations. For example, data processing resources may be available as virtual machine instances (“instances”). The instances may be configured to execute applications, including World Wide Web (“Web”) servers, application servers, media servers, database servers, and the like. The resources provided by a PES can typically be purchased and utilized according to various financial models.
A PES might expose an application programming interface (“API”) for managing the operation of instances executing within the PES. For instance, an API might be exposed for use by customers of the PES that allows the customers to create, configure, execute, and maintain instances executing on the PES. Other types of APIs might also be provided.
The methods exposed by an API for creating and maintaining instances executing within a PES might be implemented by a large number of inter-dependent components that execute in combination to provide the API. These components might be developed and/or maintained by many different development teams. Components developed and maintained by each development team might have complex dependencies upon components developed and maintained by other teams.
In order to provide optimized service to customers of the PES, it is necessary for each development team to have current information regarding the health and operation of the API components that they are responsible for. To address this, development teams typically develop and maintain their own test mechanisms for testing the API components that they are responsible for. When multiple teams develop their own testing mechanisms, however, there may be significant duplication of effort. Moreover, because test mechanisms developed by different teams are typically incompatible, it may be difficult for a development team to determine the health of dependent API components that are maintained by other development teams.
It is with respect to these and other considerations that the disclosure made herein is presented.